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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 

[Field of the Invention] 

[0001] 

This invention relates to a data perusing device, a data retrieval method, and a data retrieval program. 

[Background of the Invention] 

[0002] 

In recent years, the digital camera has spread through an ordinary home quickly. In a digital camera, 
compared with the photograph of the conventional silver halide film, a photograph can be taken easily and it 
can eliminate. Since 100 or more photographs can also be saved at one memory card, it is in the tendency 
for a lot of photographs to be taken. In the above-mentioned digital camera, since 100 or more photographs 
can be taken at a time, when using a digital camera frequently, the number of sheets of the photograph for 
one year amounts to thousands of sheets. 
[0003] 

In order to save the photograph taken in large quantities with such a digital camera, the photograph storage 
provided with HDD (hard disk), mass HDD, etc. of a personal computer etc. are used. This photograph 
storage is equipped with a high definition liquid crystal display, and there is also a product which can peruse 
a quality photograph on that spot in it. Usage of enjoying the inspection of a photograph is also carried out 
using the slide show function with which such a product was equipped, chatting with a family, a friend, etc. It 
also has directly the function which connects with a printer and prints a desired photograph. 
[0004] 

In order to make the target photograph easy to find afterwards, in such a situation, it is necessary to use 
album software, or to manage the folder to save appropriately, and to arrange beforehand. However, in a 
digital camera or photograph storage, in order to always perform such arrangement, it is taking time and 
effort for a user. 
[0005] 

then, some methods for finding a desired photograph by retrieval by keyword are proposed out of a lot of 
photographs as technology simply found by little time and effort in a lot of photographs. For example, the 
technology of enabling it to add a specific keyword to a photograph is proposed by providing a keyword 
button in a photographing instrument (for example, refer to patent documents 1). The technology which 
reduces the work of the user who adds a keyword to a picture is proposed by carrying out grouping of two or 
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more pictures, putting tlienn in blocl<, and giving a connnnent (for exannple, refer to patent docunnents 2). 

[Patent docunnents 1] JP,2002-344721,A 

[Patent docunnents 2] JP,2000-101894,A 

[Description of tine Invention] 

[Problenn to be solved by the invention] 

[0006] 

However, in the conventional technology mentioned above, in any case, a user has to add a comment at the 
time of photography or arrangement of a photograph (input), and the problem of applying a big burden to a 
user arises in respect of arrangement of a photograph after all. 
[0007] 

this invention is made in consideration of such a situation, and comes out. the purpose is to provide a data 
perusing device, a data retrieval method, and a data retrieval program which can search a photograph for 
which it asks easily out of an unsorted photograph without being alike and making troublesome operation 
perform. 

[Means for solving problem] 
[0008] 

A data storage means this invention remembers data to be in order to solve SUBJECT mentioned above, A 
displaying means which displays data memorized by said data storage means one by one, A voice input 
means which inputs a sound whenever data is displayed on said displaying means, A voice recognition 
means which carries out [ sound / which was inputted from said voice input means ] speech recognition, A 
key word storage means which relates a word in which speech recognition was carried out by said voice 
recognition means with data currently displayed on said displaying means, and memorizes it, A search term 
input means which inputs a search term, and said key word storage means and said data storage means are 
searched, A search means to specify data related with a word corresponding to a search term inputted from 
said search term input means, and a presenting means which shows said displaying means data specified by 
said search means are provided. 

When displaying data memorized by data storage means on a displaying means one by one according to this 
invention, A sound inputted from a voice input means whenever data is displayed on said displaying means. 
When carrying out speech recognition by a voice recognition means, and this word by which speech 
recognition was carried out is related with data currently displayed on said displaying means, and is 
memorized to a key word storage means and data is searched, Data which specified data related with a word 
corresponding to a search term which searches said key word storage means and said data storage means, 
and is inputted from a search term input means by a search means, and was specified by a presenting 
means is shown to said displaying means. Therefore, an advantage that a desired photograph can be 
searched easily is acquired by sound inputted at the time of an inspection, without making keyword additional 
work especially to data of a photograph etc. perform. 
[0009] 

The reading history memory measure which memorizes the reading history of the data in which this invention 
is memorized by said data storage means in the above-mentioned invention. Based on the reading history 
memorized by said reading history memory measure, provide the importance calculating means which 
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computes the importance for every data specified by said searcli means, and said presenting means, Based 
on tlie importance computed by said importance calculating means, the data specified by said search means 
is shown to said displaying means. 

According to this invention, the reading history of data is memorized to a reading history memory measure, 
the importance for every data specified by said search means based on the reading history is computed by 
an importance calculating means, and said presenting means shows data based on this importance. 
Therefore, the advantage that the photograph perused more mostly, i.e., the photograph in which the user is 
more interested, can be given priority to and shown is acquired. 
[0010] 

The printing history memory measure which memorizes the printing history of the data in which this invention 
is memorized by said data storage means in the above-mentioned invention. Based on the printing history 
memorized by said printing history memory measure, provide the importance calculating means which 
computes the importance of the data specified by said search means, and said presenting means. Based on 
the importance computed by said importance calculating means, the data specified by said search means is 
shown to said displaying means. 

According to this invention, a printing history of data is memorized to a printing history memory measure, 
importance for every data specified by said search means based on a printing history is computed by an 
importance calculating means, and said presenting means shows data based on this importance. Therefore, 
an advantage that a photograph printed more mostly, i.e., a photograph in which a user is more interested, 
can be given priority to and shown is acquired. 
[0011] 

In order to solve SUBJECT mentioned above, this invention, Speech recognition is carried out [ sound / 
which is inputted from a microphone whenever it displays data memorized by memory storage on a display 
device one by one ], When searching data which relates this word by which speech recognition was carried 
out with data currently displayed on said display device, memorizes it, and is memorized by said memory 
storage. Data related with a word corresponding to a search term inputted is searched, and searched data is 
shown to said display device. 

When according to this invention a word by which speech recognition was carried out is related with data 
currently displayed, and is memorized from a sound inputted for every data at the time of an inspection of 
data and data is searched, data related with a word corresponding to a search term inputted is searched, and 
searched data is shown. Therefore, an advantage that a desired photograph can be searched easily is 
acquired by sound inputted at the time of an inspection, without making keyword additional work especially to 
data of a photograph etc. perform. 
[0012] 

In order to solve SUBJECT mentioned above, this invention, The step which displays the data memorized on 
a display device one by one when perusing the data memorized. The step which carries out [ sound / which 
is inputted from a microphone whenever it displays data one by one ] speech recognition. When searching 
the data which makes a computer perform the step which relates this word by which speech recognition was 
carried out with the data currently displayed on said display device, and memorizes it, and is memorized, A 
computer is made to perform the step which inputs a search term, the step which searches the data related 
with the word corresponding to the search term inputted, and the step which shows said display device the 
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searched data. 

When perusing the data nnennorized according to this invention, the word by which speech recognition was 
carried out fronn the sound inputted for every data, It relates with the data currently displayed and nnennorizes, 
and when searching the data nnennorized, the data related with the word corresponding to the search ternn 
inputted is searched, and the searched data is shown. Therefore, the advantage that a desired photograph 
can be searched easily is acquired by the sound inputted at the tinne of an inspection, without nnaking the 
keyword additional work especially to the data of a photograph etc. perfornn. 
[Best Mode of Carrying Out the Invention] 
[0013] 

Hereafter, the electronic device by one ennbodinnent of this invention is explained with reference to Drawings. 

Drawing 1 is an outline view showing the appearance of the photograph storage by the ennbodinnent of this 
invention. The card slot 2, the I/field terminal 3, the display device 4, the manual operation button 5, and the 
microphone 6 are formed in the photograph storage 1. It is connected to a digital camera, and the card slot 2, 
or the I/field terminals 3 (USB etc.) incorporate the photograph photoed with this digital camera, and 
accumulate it in the built-in hard disk (after-mentioned). The photograph accumulated in the hard disk with 
which the main part was equipped with the display device 4 which consists of liquid crystal displays etc., and 
the inside was equipped can be perused now. The manual operation button 5 performs search directions of 
the photograph to peruse, inspection directions, various setting out, etc. by operation by a user. The 
microphone 6 records a user's sound during a photograph inspection, and records the word used there on 
photograph search. 
[0014] 

Next, drawing 2 is a block diagram showing the composition of the photograph storage by this embodiment. 
The same mark is attached to the portion corresponding to drawing 1 , and explanation is omitted. Although 
the main part is equipped with the display device 4, it carries out the external output of the data via a video 
output terminal etc., and it may be made to express it as an external display in a figure. The input device 10 
may be applied to the manual operation button 5 mentioned above, and may also contain a touch panel etc. 
The digital camera 30, the printer 40, etc. are connected to the l/F part 1 1 including the above-mentioned 
card slot 2, and the I/field terminal 3. 
[0015] 

The control section 12 controls the whole operation according to a predetermined program. Photograph DB 
(database)13 accumulates the photograph incorporated from the digital camera 30 via the l/F part 11. 
Photograph DB13 saves a photographing date, a file name (storing position), etc. for every ID of a 
photograph, as shown in drawing 3 . The printing directions part 14 prints the photograph saved this 
photograph DB13 with the printer 40 connected via the l/F part 11. The printing history of the printed 
photograph is recorded on the printing frequency table (after-mentioned) of index DB18 through the index 
preparing part 17 as printing frequency for every photograph. 
[0016] 

The slide show execution part 15 indicates the photograph saved photograph DB13 by a slide show with the 
display device 4. During the above-mentioned slide show execution, the voice recognition part 16 recognizes 
a user's sound inputted from the microphone 6, and extracts a word from this sound. This word lets the index 
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preparing part 17 pass, is related witli tlie pliotograph by which it is then indicated by the slide show, and is 

recorded on the key word table (after-nnentioned) of index DB18. 

[0017] 

The index preparing part 17 records the printing history of the photograph by the printing directions part 14 
on the printing frequency table (after-nnentioned) of index DB18 as printing frequency for every photograph. 
The index preparing part 17 relates with a photograph the inspection tinne which shows the display total tinne 
of each photograph to slide show execution, and records it on the inspection tinne table of index DB18. 
During slide show execution, the index preparing part 17 relates the word from the voice recognition part 16 
with the photograph by which it is then indicated by the slide show, and records it on the key word table 
(after-nnentioned) of index DB18. According to the search term (search term according [ the case of a 
microphone ] to a sound) inputted from the input device 10 or the microphone 6, the retrieval part 19 
searches index DB18 and extracts an applicable photograph from photograph DB13. 
[0018] 

Index DB18 is provided with the printing frequency table 18-1 shown in drawing 4, the inspection time table 
18-2 shown in drawing 5 , and the key word table 18-3 shown in drawing 6 . The printing frequency table 18-1 
matches and memorizes the printed number of times for every ID of a photograph, as shown in drawing 4 . 
The inspection time table 18-2 matches and memorizes the inspection time (display total time) in a slide 
show display for every ID of a photograph, as shown in drawing 5. The keyword table 18-3 matches and 
memorizes a keyword (word acquired by speech recognition during the slide show display) for every 
photograph ID, as shown in drawing 6 . 
[0019] 

Next, the operation of this embodiment mentioned above is explained. Here, drawing 7 is a flow chart for 
explaining the operation at the time of the inspection of the photograph by the photograph storage of this 
embodiment. When it judges whether there is any photograph which should be displayed on the next first at 
the time of a photograph inspection (S10) and there is the following photograph, the photograph is displayed 
with the display device 4 (SI 2), and while displaying this photograph, the surrounding sound is recorded with 
the microphone 6 (S14). Next, speech recognition is carried out [ sound / which is inputted from the 
microphone 6 ] (SI 6), the word contained in a sound is acquired (S18), and it records on the key word table 
18-3 (S20). 
[0020] 

Next, if it judges whether there were any directions of the next photograph display (S22) and there are no 
directions, it will return to Step S14 and the processing mentioned above will be repeated. That is, if a 
photograph is displayed, a user utters a word which serves as an index to the photograph toward the 
microphone 3, will be operating the manual operation button 5 after that, and will issue directions of the next 
photograph display. By this, a word which is displayed and which serves as an index for every photograph 
will be memorized. 
[0021] 

On the other hand, when there are directions of the next photograph display, it matches with the photograph 
concerned, and it returns to Step S10 which recorded inspection time on the inspection time table 18-2 
(S24), and mentioned it above, and the processing mentioned above is repeated. And the processing 
concerned will be ended if the photograph which should be displayed is lost (NO of Step S10). 
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[0022] 

Although a user's directions are performing the next photograph display in the operation at the tinne of the 
inspection nnentioned above, it nnay be a case so that the following photograph nnay be displayed for every 
fixed tinne like a slide show display. In this case, since the inspection tinne for every photograph changes, it 
nnay be nnade to record change of the inspection time for every photograph by operation of a halt etc. being 
performed by a user's directions. 
[0023] 

Next, drawing 8 is a flow chart for explaining the operation at the time of search of the photograph by the 
photograph storage of this embodiment. First, voice input of the search term is carried out from the character 
input from the input device (a touch panel is included) 10, or the microphone 6 (S40). When it inputs from the 
microphone 6, the voice recognition part 16 will perform speech recognition, and a word will be extracted, but 
explanation is omitted here. The search term "golf is inputted in the example of the graphic display. 
[0024] 

Next, the key word table 18-3 is searched, and ID of the photograph in which a keyword agrees is acquired 
(S42). At this time, also when the same keyword as two or more photographs is registered, it may be. In this 
case, ID of all the agreeing photographs will be acquired. ID=2 and ID=3 are acquired in the example of a 
graphic display. 
[0025] 

Next, one ID is acquired from search results (one or more ID) (S44), the inspection time to this ID is acquired 
from the inspection time table 18-2 (S46), and the printing frequency to this ID is acquired from the printing 
frequency table 18-1 (S48). In the example of a graphic display, to ID=2, it becomes inspection time = 10 
seconds, and printing frequency =5 time, and becomes inspection time = 10 seconds, and printing frequency 
=0 time to ID=3. 
[0026] 

Next, according to the formula used as importance =LOG(inspection time /T1)+L0G (printing frequency), 
importance is calculated so that inspection time and printing frequency are large values, and importance may 
become high. T1 is the default display time in a slide show display. In the case of inspection time =0 or 
printing frequency =0, the clause is set to 0 for error evasion. This importance serves as an index showing a 
user's interest degree. 
[0027] 

In addition, the information in which the following users' interest degree is reflected may be used as a 
parameter for computing importance. 

(1) The method of printing 

a. The size of paper 

b. The kind of paper (a regular paper / glossy paper for photographs) 

c. Printing image quality (a standard/high definition) 
[0028] 

(2) Editing history 

a. Image quality correction, a retouch 

b. A greeting card, a frame 
[0029] 
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(3) External output 

a. Memory card output 

b. Network transmission 

c. External terminal connection (inspection on a big screen display) 
[0030] 

(4) Voice features 

a. The size of voice 

b. Time when the sound is inputted 

c. Audio intonation 

d. Audio existence 
[0031] 

After calculation of importance finishes, when it judges whether importance was calculated or not about all 
the ID (S52) and unsettled ID remains in search results, it returns to Step S44 and the processing mentioned 
above is repeated. Thereby, the importance of all the ID which is search results is calculated. In the example 
of a graphic display, each importance is set to 0.30 to 1 .00 and ID=3 to ID=2. 
[0032] 

On the other hand, when importance is calculated about all the ID, the file name of a photograph is displayed 
on the display device 4 in order of importance. In the example of a graphic display, the file name 
"DSC00003.jpg" of the photograph corresponding to file name [ of the photograph corresponding to ID=2 ] 
"DSC00002.jpg" and ID=3 is displayed in an order from a top. 

[0033] 

Although speech recognition is performed and retrieval by keyword with a word is performed in the retrieving 
operation mentioned above, an index is created and it may be made to refer to audio wave shape, without 
carrying out speech recognition. 
[0034] 

By according to the embodiment mentioned above, acquiring a keyword from a user's sound by the voice 
recognition part 16, and saving as a keyword of the photograph currently perused index DB18 at the time of 
a photograph inspection, A desired photograph can be searched with the sound inputted at the time of an 
inspection, without making the keyword additional work especially to a photograph perform. A user can give 
priority to and present a more interested photograph by computing the importance of the photograph at the 
time of searching according to inspection time, a printing history, etc. 
[0035] 

In the embodiment mentioned above, although it was search to a photograph, it may apply to search of not 

only this but a video picture etc. 

[0036] 

In the embodiment mentioned above, the process of a series of processings by the printing directions part 
14, the slide show execution part 15, the voice recognition part 16, the index preparing part 17, the retrieval 
part 19, etc. which were mentioned above may be realized by the program execution by the control section 
12. These programs are memorized by the recording medium in which computer reading is possible, and 
when the control section 12 reads and executes this program, the above-mentioned processing is performed. 
Namely, the printing directions part 14, the slide show execution part 15, the voice recognition part 16, the 
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index preparing part 17, tlie retrieval part 19, etc., It realizes by central processing units', such as CPU's, 
reading the above-nnentioned progrann to nnain nnennory units, such as ROM and RAM, and perfornning 
processing and data processing of infornnation. 
[0037] 

The recording nnediunn in which connputer reading is possible here refers to a nnagnetic disk, a nnagneto- 
optical disc, CD-ROM, DVD-ROM, semiconductor nnennory, etc. This connputer progrann is distributed to a 
connputer by a connnnunication line, and the computer which received this distribution may be made to 
execute the program concerned. 
[Brief Description of the Drawings] 
[0038] 

[Drawing 1] lt is an outline view showing the appearance of the photograph storage by the embodiment of this 
invention. 

[Drawi ng 2] lt is a block diagram showing the composition of the photograph storage by this embodiment. 
[Drawing 3] lt is a key map showing the data configuration of the photograph DB. 

[Drawing 4] lt is a key map showing the data configuration of the printing frequency table of the index DB. 
[Drawing 5] lt is a key map showing the data configuration of the inspection time table of the index DB. 
[Drawing 6] It is a key map showing the data configuration of the key word table of the index DB. 
[Drawing 7] lt is a flow chart for explaining the operation at the time of the inspection of a photograph. 
[Drawing 8]lt is a flow chart for explaining the operation at the time of search of a photograph. 
[Explanations of letters or numerals] 
[0039] 

1 ~ photograph storage and 2 - a card slot and 3 -- I/field terminal, and 4 - a display device (a displaying 
means.) A presenting means, 5 ~ A manual operation button, 6 ~ Microphone (a voice input means, search 
term input means), 10 ~ An input device (search term input means), 11 - An l/F part, 12 - Control section 
(presenting means), 13 - The photograph DB (data storage means), 14 ~ A printing directions part, 15 ~ 
Slide show execution part, 16 ~ A voice recognition part (voice recognition means), 17 ~ An index preparing 
part, 18 ~ Index DB, 18-1 [ - A retrieval part (a search means, importance calculating means), 30 / ~ A 
digital camera, 40 / - Printer ] ~ A printing frequency table (printing history memory measure), 18-2 ~ An 
inspection time table (reading history memory measure), 18-3 ~ A key word table (key word storage means), 
19 



[Translation done.] 
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